Interactive segmentation

ABSTRACT

A method for three-dimensional interactive segmentation, including: receiving a three-dimensional medical image of an interior volume of a patient&#39;s body; automatically performing three dimensional segmentation on the three dimensional medical image to detect and define a region of interest, wherein the performing of the three dimensional segmentation comprises automatically determining a boundary defining the region of interest; receiving from a user spatial information indicating one or more regions of disagreement in the three-dimensional medical image with respect to the determined boundary; and updating the three dimensional segmentation of the three dimensional medical image based on the spatial information received from the user, wherein the updating comprises updating the determined boundary based on the spatial information to redefine the area of interest.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication No. 62/049,006, filed Sep. 11, 2014 and entitled“Interactive Segementation”, the contents of which are incorporatedherein by reference in their entirety.

BACKGROUND

The present invention relates to the field of computer vision and tomedical imaging analysis in particular.

Image segmentation is fundamental to medical imaging analysis andtherefore actively studied, while numerous approaches exist. Recenttrends focus on fully automatic segmentation frameworks, being muchfaster than manual annotation, less biased and repeatable. Usually, therequired workload for processing and analyzing large datasets is farbehind the ability of a human rater. Moreover, the computationaladvancements of the machine in cases that require modality fusion orthree-dimensional (3D) visualization cannot be competed even by anexpert. Nevertheless, as the outcome of the image analysis process mighthave critical implications on the patient recuperation prospects theexpertise of a clinician must be considered.

Interactive segmentation (IS) approaches may be classified based on theform and the type of input provided by the user as well as theunderlying segmentation framework (see: F. Zhao, and X. Xie. An overviewof interactive medical image segmentation. Annals of the BMVA, 7:1-22,2013). The pioneering IS work, which led to the development of the livewire technique or intelligent scissors (independently suggested by L.Dice. Measure of the amount of ecological association between species.Ecology, 26(3):297-302, 1945 and E. N. Mortensen and W. A. Barrett.Interactive segmentation with intelligent scissors. Graph. Models andImage Proc., 60(5): 349-384, 1998) is based on the image edge map. Theshortest paths between the user's mouse clicks, calculated by theDijkstra algorithm form the contour of the region of interest (ROI). Inthe united snakes (see: J. Liang, T. McInerney, and D. Terzopoulos.United snakes. 10(2):215-233, 2006), which relies on a classical activecontour framework known as snakes (see: M. Kass, A. P. Witkin, and D.Terzopoulos. Snakes: Active contour models. International Journal ofComputer Vision, 1(4):321-331, January 1988), the user ‘plants’ anchorsor seed points along the desired boundary, providing guidance for thesegmentation.

Pointing device scribbles seem to be the most common form of userinteraction. Marked regions (e.g., by pointing device scribbles) mayprovide information about the ROI and the background intensitydistributions. A well known IS approach is the GrabCut technique (see:C. Rother, V. Kolmogorov, and A. Blake. Grabcut: Interactive foregroundextraction using iterated graph cuts. SIGGRAPH, 2004) which is based onthe graph-cut (see: Y. Boykov, O. Veksler, and R. Zabih. Fastapproximate energy minimization via graph cuts. PAMI, 23(11):1222-1239,2001). Representing the image pixels by nodes in a graph, the graph-cutaddresses a foreground-background image segmentation by solving amin-cut, max-flow problem. The user's annotated regions are assigned toeither the source or the sink of the graph. In a recent paper by C.Nieuwenhuis and D. Cremers. Spatially varying color distributions forinteractive multilabel segmentation. PAMI, 35(5): 1234-1247, 2013,marked user regions, via mouse scribbles were used for gatheringspatially varying color statistics for multi-label segmentation.Level-set based segmentation framework with User Interface (UI) whichare designed for medical images were suggested in Y. Gao, R. Kikinis, S.Bouix, M. E. Shenton, and A. Tannenbaum. A 3D interactive multi-objectsegmentation tool using local robust statistics driven active contours.Medical image analysis, 16(6):1216-1227, 2012, and P. Karasev, I.Kolesov, K. Fritscher, P. Vela, P. Mitchell, and A. Tannenbaum.Interactive medical image segmentation using PDE control of activecontours. 2013. Other recent IS techniques include J. S. Prassni, T.Ropinski, and K Hinrichs. Uncertainty-aware guided volume segmentation.Visualization and Computer Graphics, 16(6):1358-1365, 2010; W. Yang, J.Cai, J. Zheng, and J. Luo. User-friendly interactive image segmentationthrough unified combinatorial user inputs. TMI, 19(9):2470-2479, 2010;L. Paulhac, J-Y. Ramel, and T. Renard. Interactive segmentation of 3Dimages using a region adjacency graph representation. In Image Analysisand Recognition, pages 354-364. 2011 and K. McGuinness and N. E.OConnor. Toward automated evaluation of interactive segmentation. CVIU,115(6):868-884, 2011.

The foregoing examples of the related art and limitations relatedtherewith are intended to be illustrative and not exclusive. Otherlimitations of the related art will become apparent to those of skill inthe art upon a reading of the specification and a study of the figures.

SUMMARY

The following embodiments and aspects thereof are described andillustrated in conjunction with systems, tools and methods which aremeant to be exemplary and illustrative, not limiting in scope.

There is provided, in accordance with an embodiment, a method comprisingusing at least one hardware processor for: receiving a three-dimensionalmedical image of an interior volume of a patient's body; automaticallyperforming three dimensional segmentation on the three dimensionalmedical image to detect and define a region of interest in the threedimensional medical image, wherein the performing of the threedimensional segmentation comprises automatically determining a boundarydefining the region of interest; receiving an input from a user withrespect to the three dimensional segmentation of the three dimensionalmedical image, wherein the input comprises spatial informationindicating one or more regions of disagreement in the three-dimensionalmedical image with respect to the determined boundary; and updating thethree dimensional segmentation of the three dimensional medical imagebased on the input received from the user, wherein the updatingcomprises updating the determined boundary based on the spatialinformation to redefine the area of interest, and wherein the updatingis performed according to weights determining the relative contributionof each one of the three dimensional segmentation and the input from theuser to the updating of the three dimensional segmentation, therebydetermining the extent of influence of the input from the user on thedetermined boundary.

There is provided, in accordance with another embodiment, a computerprogram product for 3D interactive segmentation, the computer programproduct comprising a non-transitory computer-readable storage mediumhaving program code embodied therewith, the program code executable byat least one hardware processor to: automatically perform threedimensional segmentation on the three dimensional medical image todetect and define a region of interest in the three dimensional medicalimage, wherein the performing of the three dimensional segmentationcomprises automatically determining a boundary defining the region ofinterest; receive an input from a user with respect to the threedimensional segmentation of the three dimensional medical image, whereinthe input comprises spatial information indicating one or more regionsof disagreement in the three-dimensional medical image with respect tothe determined boundary; and update the three dimensional segmentationof the three dimensional medical image based on the input received fromthe user, wherein the updating comprises updating the determinedboundary based on the spatial information to redefine the area ofinterest, and wherein the updating is performed according to weightsdetermining the relative contribution of each one of the threedimensional segmentation and the input from the user to the updating ofthe three dimensional segmentation, thereby determining the extent ofinfluence of the input from the user on the determined boundary.

There is provided, in accordance with a further embodiment, a medicalsystem comprising: a medical imaging device; at least one hardwareprocessor configured to: i) automatically perform three dimensionalsegmentation on the three dimensional medical image to detect and definea region of interest in the three dimensional medical image, wherein theperforming of the three dimensional segmentation comprises automaticallydetermining a boundary defining the region of interest; ii) receive aninput from a user with respect to the three dimensional segmentation ofthe three dimensional medical image, wherein the input comprises spatialinformation indicating one or more regions of disagreement in thethree-dimensional medical image with respect to the determined boundary;and iii) update the three dimensional segmentation of the threedimensional medical image based on the input received from the user,wherein the updating comprises updating the determined boundary based onthe spatial information to redefine the area of interest, and whereinthe updating is performed according to weights determining the relativecontribution of each one of the three dimensional segmentation and theinput from the user to the updating of the three dimensionalsegmentation, thereby determining the extent of influence of the inputfrom the user on the determined boundary.

In some embodiments, the three dimensional medical image is of animaging modality type selected from the group consisting of:Computerized Tomography (CT), Magnetic Resonance Imaging (MM) andultrasound.

In some embodiments, the method further comprises using said at leastone hardware processor for performing three dimensional visualization ofthe segmented region of interest.

In some embodiments, the method further comprises using said at leastone hardware processor for estimating at least one measurable feature ofthe region of interest.

In some embodiments, the at least one measurable feature is selectedfrom the group consisting of: volume, length, width, height, pose,location and texture.

In some embodiments, the receiving of the input from the user and theupdating of the three dimensional segmentation are performed in aniterative manner.

In some embodiments, the receiving of the input from the user isperformed via a Graphical User Interface (GUI).

In some embodiments, the GUI presents to the user one or more views ofthe 3D medical image selected from the group consisting of: slice viewsin one or more body planes and a 3D view.

In some embodiments, the spatial information comprises anchors indicatedby the user via a pointing device on the one or more views of the 3Dmedical image presented by the GUI.

In some embodiments, the automatically performing of the threedimensional segmentation on the three dimensional medical image todetect and define the region of interest and the updating of the threedimensional segmentation of the three dimensional medical image based onthe input received from the user comprises using a probabilistic model.

In some embodiments, the probabilistic model is based on GaussianMixture Model (GMM), and wherein the GMM is used to model a distributionof the intensities of the 3D medical image within and outside of theregion of interest.

In some embodiments, the performing of the three dimensionalsegmentation further comprises defining a cost functional, the costfunctional comprising: an image likelihood term representing the threedimensional segmentation, and a user interaction term representing theinput from the user, assigning the weight of the weights determining therelative contribution of the three dimensional segmentation to the imagelikelihood term, assigning the weight of the weights determining therelative contribution of the input from the user to the user interactionterm, and setting the weight of the weights assigned to the userinteraction term to zero.

In some embodiments, the performing of the three dimensionalsegmentation and the updating the three dimensional segmentation furthercomprise finding a segmentation that optimizes the cost functional byusing a gradient descent process.

In addition to the exemplary aspects and embodiments described above,further aspects and embodiments will become apparent by reference to thefigures and by study of the following detailed description.

BRIEF DESCRIPTION OF THE FIGURES

Exemplary embodiments are illustrated in referenced figures. Dimensionsof components and features shown in the figures are generally chosen forconvenience and clarity of presentation and are not necessarily shown toscale. The figures are listed below.

FIG. 1 shows a flowchart of a method for performing a 3D interactivesegmentation, constructed and operative in accordance with an embodimentof the disclosed techniques;

FIG. 2A shows a screen shot of an exemplary Graphical User Interface(GUI) presenting views of a segmented 3D medical image;

FIG. 2B shows a screen shot of the GUI of FIG. 2A including a user inputwith respect to the segmentation of the 3D medical image; and

FIGS. 3A-3D show various views of a Computerized Tomography (CT) 3Dmedical image of CH in a human brain of various patients segmentedaccording to the disclosed probabilistic model.

DETAILED DESCRIPTION

3D interactive segmentation of medical images is disclosed herein, whichprovides an automated segmentation algorithm integrated with contextualknowledge, not readily available in the images alone, provided by anexperienced physician or another human handler (jointly referred toherein as a “user”). Thus, the disclosed 3D interactive segmentation mayprovide more accurate segmentation results. Image segmentation innecessary for measuring ROI features, such as size (e.g., volume), pose,location, texture, etc., which may be critical for diagnosis, treatmentplanning and image-guided therapy. User interaction may be essential forresolving classification ambiguities and errors due to imagingartifacts, poor contrast and noise. The disclosed 3D interactivesegmentation of medical images may be utilized, for example, for tumorsegmentation for, e.g., fine surgeries, focus ultrasound treatment andthe like.

The disclosed 3D interactive segmentation may support an intuitive,convenient and friendly user interaction subjected to the bottom-upconstraints introduced by the image intensities. The user may perform a“dialogue” with the segmentation algorithm, for example, via severalpointing device inputs in regions of disagreement, formulated as anadditional, spatial term in a global cost functional for 3Dsegmentation. As no prior information may be assumed and the imageintensity distribution may be learned throughout the segmentation, theproposed method may be used for a wider range of segmentationapplications and imaging modalities.

The disclosed 3D interactive segmentation may provide a user interactiveframework for the segmentation of volumetric ROIs. Optionally, alevel-set framework may be used, as it is parameterization-free, mayallow automatic topological changes and may enable a straightforwardgeneralization to high-dimension and multi-label segmentation.Segmentation may be obtained by solving a maximum a posteriori (MAP)problem. Using calculus of variation, an optimum of a cost functionalmay be searched, which is based on a generative probabilistic model. Aninitial, fully automatic segmentation process may be dominated by thegray level distributions of the ROI and the background partitions. Auser input, which may be provided by a pointing device or any otherinput device, may add spatial constraints to the segmentation problem.These constraints are soft and may express, what is termed here theuser's certainty disagreement map. The user's disagreement map may beconvolved with a Gaussian kernel which may define the local extent ofthe user influence and may control the level of disagreement (or user'sconfidence). This contribution may allow establishing a user-machinedialogue. Optionally, the user may not be allowed to edit thesegmentation. Instead, the model may provide a framework to refinesegmentation by resolving voxel annotation ambiguities, through“negotiation”.

The term “medical imaging device” as referred to herein, may relate to adevice configured to create visual representations of the interior of abody of a patient for clinical analysis and/or medical intervention.Such device may utilize an imaging modality including ComputerizedTomography (CT), Magnetic Resonance Imaging (MM), ultrasound, and/or thelike.

The term “3D medical image” or “3D image” as referred to herein, mayrelate to a set of multiple two-dimensional (2D) scans of differentslices of a desired interior volume in a patient's body in multiple bodyplanes (e.g., coronal, sagittal and axial) performed by a medicalimaging device. The set of multiple 2D scans may be then combined toproduce a 3D model of the interior volume by dedicated software.However, it is intended herein explicitly that the term “3D medicalimage” covers also non-medical images, such as various industrial imagesthat depict an internal volume of a certain object.

The term “patient” as referred to herein, may relate to any livingcreature including human beings and animals.

The term “region” as referred to herein, may relate to a 2D region or toa 3D region, according to the specific context. For example, whenreferring to a region in a scan, which is 2D, then the region is a 2Dregion. However, when referring to a region in a 3D image or a 3D model,then the region is a 3D region.

The term “region of interest” as referred to herein with respect to ascan or an image (i.e., 3D medical image), may include one or moreseparate regions in the medical scan or image, which region(s) areexpected to encompass an anatomical structure, such as the skull or atissue, or an abnormality, such as a healthy tissue distorted due to thepresence of a tumor, a hemorrhage, a tumor and/or the like.

The term “boundary” as referred to herein with respect to an ROI definedin a medical image or a scan, may relate to a boundary delimiting theROI in the medical image or the scan. Such boundary may include eachseparate boundary which delimits each separate region of the ROI. Theboundary may be a line consisting of pixels (e.g., when defined in a 2Dscan) or of a surface consisting of voxels (e.g., when defined in a 3Dimage).

Reference is now made to FIG. 1, which shows a flowchart of a method forperforming a 3D interactive segmentation, constructed and operative inaccordance with an embodiment of the disclosed techniques.

In a step 100, a 3D medical image of an interior volume of a patient'sbody may be received. The 3D medical image may be of an imaging modalitytype, such as Computerized Tomography (CT), Magnetic Resonance Imaging(Mill) and ultrasound. The 3D medical image may include a set ofmultiple two-dimensional (2D) scans of different slices of the interiorvolume.

In a step 110, 3D segmentation may be automatically performed on the 3Dmedical image to detect and define a region of interest (ROI) in the 3Dmedical image. The segmentation may include automatically determining aboundary around the ROI. Various methods of the prior art may be used toperform the 3D segmentation. Alternatively, the segmentation method laidout below may be used.

In some embodiments, the segmentation may be performed by estimating theprobability of voxels of the 3D medical image to be within or outside ofthe ROI, where each voxel may be characterized by intensity.Accordingly, the automatically performing of the 3D segmentation mayinclude using a probabilistic model. The probabilistic model may bebased on Gaussian Mixture Model (GMM). The GMM may be used to model adistribution of the intensities of the 3D medical image within andoutside of the ROI. The automatic performing of the 3D segmentation mayinclude defining a cost functional. The cost functional may include animage likelihood term representing the three dimensional segmentationand a user interaction term representing the user input. Optionally, thecost functional may further include a regularization term representingsegmentation smoothness constraints.

The cost functional may be formulated according to a level-set frameworkand in a continuous form. Weights may be assigned to the imagelikelihood term and to the user interaction term determining therelative contribution of each term to the 3D segmentation. Theregularization term may be also assigned a weight. Accordingly, at thisstage, the weight assigned to the user interaction term may be set tozero.

A segmentation which optimizes the cost functional may be found via agradient descent process. A probability map of the image voxels to bewithin or outside of the ROI may be then determined. In someembodiments, a probability threshold may be defined in order todeterministically define the ROI. This may be performed by comparing theprobability of each voxel to the threshold in order to determine if thevoxel belongs to the ROI or to the background. Then, a boundary of theROI, resulting from the application of the threshold, may be displayedto the user. Optionally, the boundary is shown superimposed on themedical image. The probability threshold may be set, for example, to0.5.

In a step 120, an input with respect to the 3D segmentation of the ROImay be received from a user. The user may advantageously be a physicianor any other medical specialist. The input may include spatialinformation, e.g., indicating one or more regions of disagreement in thethree-dimensional medical image with respect to the determined boundary.Thus, for example, the user may disagree with the determination (e.g.,according to the automatically performed 3D segmentation of step 110)that a specific region is included in the background or in the ROI(i.e., as defined by the determined boundary). The spatial informationprovided by the user may not be accurate, in order to not burden theuser. Thus, the user may indicate regions and not necessarily exactlocations. The user input may add spatial constraints to the fullyautomatic segmentation process of step 110, which may be dominated bythe gray level distributions of the 3D image. Advantageously, since theuser input is spatial (as opposed to input relating to the grey levels),it may influence the automatic segmentation process based on the greylevel distribution of the 3D image in only a local and specific manner.

In some embodiments, the input from the user may be received via a GUI.Reference is now made to FIG. 2A, which shows a screen shot of anexemplary GUI 200, which presents views of a segmented 3D medical image.The 3D medical image is of a human brain which includes a cerebralhemorrhage (CH). The CH may be segmented (i.e., ROI) according to step110. GUI 200 may include one or more views of or deriving from the 3Dmedical image, such as slice views 210, 220 and 230 and a 3D view 240.Slice views 210, 220 and 230 may present to the user the 2D scans of thedifferent slices of the interior volume. Slice view 210 may presentscans in X-Y body planes (i.e., coronal planes), slice view 220 maypresent scans in Y-Z body planes (i.e., sagittal planes) and slice view230 may present scans in X-Z body planes (i.e., axial planes). 3D view240 may present a 3D model 300 of the segmented area of interest.Alternatively or additionally, 3D view 240 may present a 3D model of theentire interior volume, and/or of other portions of the interior volume.

GUI 200 may include notations 215, 225, 235 and 245 for slice views 210,220 and 230 and 3D view 240, correspondingly, which may indicate therelative position of elements in the scans or of the 3D model within the3D medical image. GUI 200 may further include a 3D scroll bar 250 whichmay allow the user to scroll through scans in a selected body plane. 3Dscroll bar may include X scroll bars 252, Y scroll bars 254, and Zscroll bars 256. X scroll bars may include a left scroll bar 252A and aright scroll bar 252B. Y scroll bars may include a left scroll bar 254Aand a right scroll bar 254B. Z scroll bars may include an up scroll bar256A and a down scroll bar 256B. For example, the user may scrollthrough all of the 2D scans of the 3D medical image which are in acoronal plane by clicking Z bars 256. The user may go up along an axis Zof the 3D medical image by clicking the up scroll bar 256A or down alongthe axis by clicking down scroll bar 256B. The scans may be shown inslice view 210. GUI 200 may further include a position tag 310, whichmay indicate a current position of each one of the 2D scans shown inslice views 210 (along the z axis), 220 (along the x axis) and 230(along the y axis).

GUI 200 may further allow the user to rotate 3D model 300 in a pluralityof directions, optionally in all directions. The user may rotate 3Dmodel 300 by an input device such as a mouse.

In some embodiments, the spatial information may include anchorsindicating a spatial region within the 3D medical image. The anchors maybe indicated on one or more views of the 3D medical image presented byGUI 200. Reference is now made to FIG. 2B, which shows a screen shot ofGUI 200 of FIG. 2A including a user input with respect to thesegmentation of the 3D medical image. The segmentation of the 3D medicalimage according to step 110 may be presented to the user by a boundaryline 320, for example, of a specific color, segmenting the CH. The usermay then define a region of disagreement by anchors, which may beindicated, for example, via a pointing device (e.g., a mouse or atouchpad) by pointing on one or more views of the 3D medical image, orvia any other input device, such as a keyboard or a touch screen. Forexample, anchors may be located by the user on a view of the 3D medicalimage to indicate a region to be added to the segmented ROI (i.e., aregion of disagreement). In this manner, the user may indicate that thedetermined boundary should also delimit the region defined by thelocated anchors. As another example, anchors may be located to indicatea region to be removed from a segmented ROI (i.e., a region ofdisagreement). In this manner, the user may indicate that the determinedboundary should not delimit the region defined by the located anchors.The anchors may be represented by GUI 200 as asterisks 330 and maydiffer, for example, by color or shape, in order to distinguish betweenanchors which define a region to be added to the segmented ROI andanchors which define a region to be removed from the segmented ROI.

GUI 200 may further include user interaction buttons 260, a results tag270, control buttons 280 and a status tag 290. User interaction buttons260 may allow the user to interact or have a ‘dialogue’ with thesegmentation process. For example, the user may indicate that thesegmentation is satisfying by pressing a ‘satisfied’ button 262, add aninput (i.e., according to step 120) by pressing input buttons 264 and266 or update the segmentation of the 3D medical image (i.e., accordingto step 130 below) by pressing an update button 268. Input button 266may allow the user to add anchors to the segmented 3D image (e.g., bylocating asterisks or plus signs on the presented 2D scans of the 3Dmedical image). Input button 264 may allow the user to add a linedefined by a sequence of anchors. Results tag 270 may present to theuser the values of measured features of the defined ROI (see step 140below), such as volume or other size dimensions (i.e., length, width andheight) pose (i.e., the exact location of the center of mass), location,texture (i.e., by applying a sequence of filters which may producevalues that may characterize particular textures), etc. User controlbuttons 280 may include buttons which allow the user to perform variousoperations such as load a 3D medical image by pressing a load button 282(e.g., by clicking or touching on an input device or touching the buttonitself as presented on the screen), start a segmentation process bypressing a start button 284 and exit the GUI and the segmentationprocess by pressing on an exit button 286. Status tag 290 may show thestatus of the entire segmentation process, which may be an iterativeprocess, e.g., by indicating that segmentation (i.e., a segmentationiteration) is complete. GUI 200 may additionally allow the user toenlarge or reduce slice views 210, 220 and 230 and/or 3D view 240.

In a step 130, the 3D segmentation of the region of interest may beupdated based on the input received from the user. For example, and withreference to FIG. 2A, the user may press update button 268 in order toinitiate or execute the update process. Alternatively, the updatingprocess may be initiated automatically once an input from the user isreceived. Optionally, the update may be performed according to theGMM-based probabilistic model. At this stage, the weight assigned to theuser interaction term may be set to a predefined or a selected valuehigher than zero. The weight assigned to the user interaction term maydetermine the extent of the user's influence, i.e., the influence of theuser's input on the automatic segmentation. The user may not edit thedetermined boundary. Such an approach takes into account human error andis more user friendly since it may allow the user to provide spatialinformation which is not necessarily accurate. This weight value of theuser input at this stage may be selected by the user or may bepredefined.

Optionally, the weight assigned to the regularization may be set, atthis stage, to a value which is lower than its value at the automaticsegmentation stage (i.e., step 110) or may be set to zero. Optionally,the user input may be integrated with the determined 3D segmentation ina probabilistic manner according to the probabilistic model indicatedabove. A 3D segmentation which may optimize the cost functional may befound by using a gradient descent process. A more detailed descriptionof the updating of the 3D segmentation according to this technique isprovided herein below.

In some embodiments, the receiving of the input from the user withrespect to the 3D segmentation of the region of interest and theupdating of the 3D segmentation of the region of interest based on theinput received from the user may be performed in an iterative manner.Thus, the user may keep providing input and following that, thesegmentation of the ROI may be updated until the user is satisfied fromthe results. The iterations may allow the user to amend his errors.Furthermore, the iterations may allow the user, ultimately, to provide amore accurate input achieved in a manner which is more convenient anduser-friendly. In addition, the iterations may allow the user to bestowhis input more weight than originally given, in case the weight of theuser input is predefined.

At the first iteration, a first fully automatic segmentation isperformed according to step 110. A first update according to a firstuser input may be performed with respect to the fully automaticsegmentation (according to step 130). At the next iterations, the updateis performed with respect to the current determined segmentation (i.e.,as previously updated). According to experimental results, the overallnumber of interaction steps needed was in general not higher than two,but further iterations, if needed, may be performed.

In an optional step 140, measurable features of the ROI may beestimated. Such features may include: size features such as volume,pose, location, texture, etc. The estimation of such features may bepresented to the user, for example, in results tag 270 of GUI 200 ofFIG. 2A.

In an optional step 150, a 3D visualization of the segmented ROI may beperformed. For example, via GUI 200 and such as 3D model 300.

A medical system for performing interactive 3D segmentation is hereindisclosed. The medical system may include a medical imaging device andat least one hardware processor. The medical imaging device may beconfigured to capture the 3D medical image of an interior volume of apatient's body and the at least one hardware processor may be configuredto interactively segment a region of interest of the captured 3D medicalimage according to the method of FIG. 1 and the techniques disclosedherein. The medical system may further include a non-transitory storagemedium which may store the captured 3D medical image. The medicalimaging device may be as known in the art, and such as a CT device, anMM device or an ultrasound device.

A probabilistic model based on GMM, which may be used for 3D interactivesegmentation of the ROI, is disclosed herein below. At first, adefinition and formulation of the problem presented by the disclosed 3Dinteractive segmentation is described.

Let I be a gray-level image defined on a 3D image domain Ω. The goal isto find the ROI/background image partitions denoted by ω and Ω\ω,respectively. Let S:Ω→{0,1} denote the unknown binary voxel annotationof I corresponding to this partition. One may assume that the observedimage I is generated such that each image voxel is independent andidentically distributed random variable (i.i.d) by the respectivesegmentation S with probability p(I|S; ψ) where ψ are GMM parameters ofimage intensities. A fully automatic segmentation may be stated as a MAPproblem:

p(S|I;ψ)∝(I|S;ψ)p(S),  (1)

where the prior p(S) is used for regularization as one shall see in thefollowing. Incorporating the user input, the proposed segmentationframework may be carried out in a step-by-step manner, where at eachstep k the user provides a set of spatial parameters through theinteraction, after observing the previous segmentation estimate, i.e.S^(k-1). This set may be denoted by η^(k) and one may assume that S^(k)is generated with probability p(S^(k)|S^(k-1);η^(k)). One should notethat the similarity of this conditional probability to a first orderMarkov chain where η^(k) may be considered as the transition probabilityparameters. Given S^(k-1), the intensity distribution parameters of theobserved image, denoted by and the user input, i.e. η^(k), one maydefine the posterior probability of S^(k), using the Bayes theorem andthe chain rule, as follows:

p(S ^(k) |I,S ^(k-1);ψ,η^(k))∝p(I|S ^(k);ψ)p(S ^(k) |S^(k-1);η^(k)).  (2)

One may look for Ŝ^(k)=argmax_(s) _(k) p(S^(k)|I,S^(k-1);ψ,η^(k)) byminimizing the following cost functional:

ε=−log p(I|S ^(k);ψ)−log p(S ^(k) |S ^(k-1);η^(k)),  (3)

using the proportion ε∝−log p. In the following, an explicit formulationof the image likelihood term p(I|S^(k);ψ) to and the user input termp(S^(k)|S^(k-1);η) will be presented. One may assume i.i.d. distributionof the image voxels. Therefore the probability over the entire imagedomain will be presented by the product of probabilities of each voxel.A continuous form of ε, based on a level-set framework will beintroduced next, followed by a gradient descent optimization process toestimate S^(k).

Mixtures of Gaussians may be used to model the intensity distributionwithin and outside the ROI. Let ψ_(in)={μ_(i) ^(in),σ_(i) ^(in),w_(i)^(in)}_(i=1) ^(N) ^(in) and let ψ_(out)={μ_(i) ^(out),σ_(i) ^(out),w_(i)^(out)}_(i=1) ^(N) ^(out) , where N^(in) and N^(out) are the respectivenumbers of Gaussians and w_(i) are the weights. For each image voxel xwith intensity I(x) one may define:

${P_{in}(x)} \propto {\sum\limits_{i = 1}^{N^{in}}{w_{i}^{in}{\exp ( {- \frac{( {{I(x)} - \mu_{i}^{in}} )^{2}}{( \sigma_{i}^{in} )^{2}}} )}\mspace{14mu} {and}}}$${{P_{out}(x)} \propto {\sum\limits_{i = 1}^{N^{out}}{w_{i}^{out}{{\exp ( {- \frac{( {{I(x)} - \mu_{i}^{out}} )^{2}}{( \sigma_{i}^{out} )^{2}}} )}.}}}}\;$

The image likelihood cost, for a given ROI, may be defined as follows:

$\begin{matrix}\begin{matrix}{ɛ_{IL} = {{- {\sum\limits_{x\; \in \omega^{k}}{\log \; {P_{in}( {{I(x)},\psi_{in}} )}}}} - {\sum\limits_{x\; \in {\Omega \backslash \omega^{k}}}{\log \; {P_{out}( {{I(x)},\psi_{out}} )}}}}} \\{= {- {\sum\limits_{x \in \Omega}\lbrack {{{S^{k}(x)}\log \; {P_{in}( {{I(x)},\psi_{in}} )}} + {( {1 - {S^{k}(x)}} )\log \; {P_{out}( {{I(x)},\psi_{out}} \rbrack}}} }}}\end{matrix} & (4)\end{matrix}$

One should note that both the ROI (or equivalently, the segmentationS^(k)) and the intensity parameters ψ are unknown, and are thereforeestimated via an alternating minimization scheme which will be detailedin the following.

For example, by using mouse clicks, the user may indicate disagreementwith the current segmentation. The collection of user clicks at a timestep k may be considered as a forest of impulse responses U^(k)=Σ_(m=1)^(M) ^(k) δ(x_(m) ^(k)) defined on the 3D image domain, where x_(m)^(k)=(x_(m) ^(k),y_(m) ^(k),z_(m) ^(k)) are the coordinates of a markedvoxel and M^(K) is the number of clicks at time step k. Let {circumflexover (ω)}^(k-1) be the ROI estimate at step k−1. One may then definep(S^(k-1)(x))

p(xε{circumflex over (ω)}^(k-1)). To the first approximation, thetransition probability, given the user input may take the followingform:

$\begin{matrix}{{p( { S^{k} \middle| S^{k - 1} ;U^{k}} )} = \{ {\begin{matrix}{p( S^{k - 1} )} & {{{if}\mspace{14mu} U^{k}} = 0} \\{1 - {p( S^{k - 1} )}} & {{{if}\mspace{14mu} U^{k}} = 1}\end{matrix}.} } & (5)\end{matrix}$or, alternatively,

p(S ^(k) |S ^(k-1) ;U ^(k))=p(S ^(k-1))^((1-U) ^(k) ⁾(1−p(S ^(k-1)))^(U)^(k)   (6)

In practice, one may smooth the binary user input by a convolution witha Gaussian kernel: η^(k)

U^(k)*G_(σ) _(U) . The transition probability may now be rephrased asfollows:

p(S ^(k) |S ^(k-1);η^(k))=p(S ^(k-1))^((1-η) ^(k) ⁾(1−p(S ^(k-1)))^(η)^(k) .  (7)

One should note that the voxels background/foreground assignments arenot altered due to the user input. Instead, the assignment probabilityof a voxel at x may be flipped with probability η^(k)(x), where η^(k)represents the user's confidence or certainty disagreement map at stepk. The extent of the user's influence, i.e., au, may be eitherdetermined by the user or set to a default value. To avoid cases inwhich a user click within the ROI affects the background or vice versaone may define ROI-background user maps as follows: η^(k)_(in)=S^(k-1)η^(k) and η^(k) _(out)=(1−S^(k-1))η^(k). One may now definea user interaction energy term as a sum of two components referring tothe user's clicks within the ROI and outside it:

$\begin{matrix}\begin{matrix}{ɛ_{UI} = {{- {\sum\limits_{x\; \in \omega^{k}}{\log_{in}\; {p( {{S^{k - 1}(x)},{\eta_{in}^{k}(x)}} )}}}} -}} \\{{\sum\limits_{x\; \in {\Omega \backslash \omega^{k}}}{\log \; {p_{out}( {{S^{k - 1}(x)},{\eta_{out}^{(k)}(x)}} )}}}} \\{{= {- {\sum\limits_{x \in \Omega}\begin{bmatrix}{{{S^{k}(x)}\log \; {p_{in}( {{S^{k - 1}(x)},\eta_{in}^{k}} )}} +} \\{( {1 - {S^{k}(x)}} )\log \; {p_{out}( {{S^{k - 1}(x)};\eta_{out}^{k}} )}}\end{bmatrix}}}},}\end{matrix} & (8)\end{matrix}$

where, log p_(in)(S^(k-1),η^(k) _(in))

log p(S^(k)|S^(k-1);η^(k) _(in)), and log p_(out)(S^(k-1),η^(k) _(out))

log p(S^(k)|S^(k-1);η^(k) _(out)), are defined by Eq. (7).

One may use a level-set framework to formulate theuser-interface-segmentation problem (see: S. Osher and J. A. Sethian.Fronts propagating with curvature-dependent speed: Algorithms based onHamilton-Jacobi formulations. Journal of Computational Physics,79:12-49, 1988). Let φ^(k) define a level-set function, such thatδω^(k)={x|φ^(k)(x)=0} may denote the estimated ROI boundaries in I atstep k, and let ω_(k) denote the corresponding ROI domain. As in: T. F.Chan and L. A. Vese. Active contours without edges. TIP, 10(2):266-277,2001, the binary segmentation S^(k) may be represented by applying theHeaviside function to φ^(k), i.e. H(φ^(k)). Adopting the probabilisticapproach in T. Riklin Raviv, K. Van Leemput, B. H. Menze, W. M. Wells,and P. Golland. Segmentation of image ensembles via latent atlases.Medical Image Analysis, 14(5):654-665, 2010, one may use the logisticregression sigmoid, which may be used as a regularized form of theHeaviside function, to represent the soft segmentation p(S^(k)):

$\begin{matrix}{{{H_{ɛ}( \varphi^{k} )} = {{\frac{1}{2}( {1 + {\tanh( \frac{\varphi^{k}}{2ɛ} )}} )} = \frac{1}{1 + e^{{- \varphi^{k}}/ɛ}}}},} & (9)\end{matrix}$

where ε may be used to determine the fuzziness of the estimated ROIboundaries (see: T. Riklin Raviv, K. Van Leemput, B. H. Menze, W. M.Wells, and P. Golland. Segmentation of image ensembles via latentatlases. Medical Image Analysis, 14(5):654-665, 2010). As is in theaforementioned Raviv et al., one may now define the level-set functionφ^(k), as follows:

$\begin{matrix}{{\varphi^{k}(x)}\overset{\Delta}{=}{{ɛ\; {{logit}(p)}} = {{ɛ\; \log \frac{p( {x \in \omega^{k}} )}{1 - {p( {x \in \omega^{k}} )}}} = {ɛ\; \log {\frac{p( {x \in \omega^{k}} )}{p( {x \in {\Omega \backslash \omega^{k}}} )}.}}}}} & (10)\end{matrix}$

It may be shown by substitution of Eq. (10) into Eq. (9) thatH_(ε)(φ^(k)) is equivalent to p(xεω^(k)). This relation is fundamentalin the proposed probabilistic level-set framework. It is important tonote that φ^(k) is generated in an iterative manner, using a gradientdescent framework. However, k is not an iteration index (which will bedenoted by τ) but the final form of a level-set function after a firststep of fully automatic segmentation followed by k−1 steps of userinteraction. The number of steps may be not higher than two or three.Next, one may use the equivalence between p(S^(k)) and H_(ε)(φ^(k)), andthe continuous forms of the equations above to resolve the segmentationproblem via a level-set based gradient descent optimization.

The proposed cost functional for level-set based segmentation includesan image likelihood term, ε_(IL), a regularization term, ε_(REG), and auser interaction term ε_(UI):

ε(φ^(k) |I,φ ^(k-1),ψη^(k))=ε_(IL)(φ^(k)|I,ψ)+ε_(UI)(φ^(k)|φ^(k-1),η^(k))+ε_(REG)(φ^(k))  (11)

Adopting a continuous formulation and a soft definition of the ROI, thesum over the image voxels, i.e., Σ_(x)εΩ, is replaced with anintegration: ∫_(Ω)dx, and the binary segmentation S^(k)(x) is replacedby the probability that a voxel x belongs to the ROI: p(S^(k)(x)) or,using the level-set formulation, by H_(ε)(φ^(k)(x)). The explicit formof the energy functional in Eq. (11) is as follows:

$\begin{matrix}{{ɛ( \varphi^{k} )} = {{\int_{\Omega}{W_{UI}\lbrack {{{H_{ɛ}( {\varphi^{k}(x)} )}\log \; {p_{in}( \eta_{in}^{k} )}} + {{H_{ɛ}( {{- \varphi^{k}}\; (x)} )}\log \; {p_{out}( \eta_{out}^{k} )}}} \rbrack}}\  + {W_{IL}\lbrack {{{H_{ɛ}( {\varphi^{k}(x)} )}\log \; {p_{in}( {I;\psi_{in}} )}} + {{H_{ɛ}( {{- \varphi^{k}}\; (x)} )}\log \; {p_{out}( {I;\psi_{out}} )}}} \rbrack} + {W_{LEN}{{\nabla{H_{ɛ}( {\varphi^{k}(x)} )}}}{{dx}.}}}} & (12)\end{matrix}$

The last term in Eq. (12), i.e. |∇H_(ε)(φ^(k)(x))|, is known, in thelevel-set literature as the smoothness or regularization term (see: T.F. Chan and L. A. Vese. Active contours without edges. TIP,10(2):266-277, 2001). The relation to −log p(S^(k)) where p(S^(k)) isthe prior in Eq. (1) is shown in: T. Riklin Raviv, K. Van Leemput, B. H.Menze, W. M. Wells, and P. Golland. Segmentation of image ensembles vialatent atlases. Medical Image Analysis, 14(5):654-665, 2010. The weightsW_(UI), W_(IL), W_(LEN), may be positive scalars that balance thecontribution of each term. In the first, fully automatic segmentationphase W_(UI) is set to zero. Similarly, W_(LEN) may be set to a lowervalue (or zero) in the presence of user interaction.

One may look for the segmentation φ^(k) that optimizes the energyfunctional Eq. (12) via a gradient descent process:

${\varphi_{\tau + {\Delta \; \tau}}^{k} = {\varphi_{\tau}^{k} + {\Delta \; \tau \frac{\partial\varphi^{k}}{\partial\tau}}}},$

where, φ_(τ) ^(k) is the level-set estimate at iteration τ in step k.The gradient descent step

$\frac{\partial\varphi^{k}}{\partial\tau}$

may be derived from the first variation of the functional above and maydetermine the evolution of the segmentation:

$\begin{matrix}{{\frac{\partial\varphi^{k}}{\partial\tau} = {{\delta_{ɛ}( \varphi^{k} )}\{ {{W_{UI}\lbrack {{\log \; {p_{in}( \eta_{in}^{k} )}} - {\log \; {p_{out}( \eta_{out}^{k} )}}} \rbrack} + {W_{IL}\lbrack {{\log \; {p_{in}( {I;\psi_{in}} )}} - {\log \; {p_{out}( {I;\psi_{out}} )}}} \rbrack} + {W_{LEN}{{div}( \frac{\nabla\varphi^{k}}{{\nabla\varphi^{k}}} )}}} \}}},} & (13)\end{matrix}$

where, div is the divergence operator and δ_(ε)(φ^(k)) is the derivativeof H_(ε)(φ^(k)) with respect to φ^(k):

${\delta_{ɛ}(\varphi)} = {\frac{{dH}_{ɛ}(\varphi)}{d\; \varphi} = {\frac{1}{4ɛ}{{{\sec h}^{2}( \frac{\varphi}{2ɛ} )}.}}}$

The scalar maps p_(in)(η^(k) _(in)), p_(out)(η^(k) _(out)) may beupdated once after each segmentation step, if a user input is provided.The GMM of the image intensities ψ_(in) and ψ_(out) are re-estimated incorrespondence to φ_(τ) ^(k), which determines the ROI boundaries, usingexpectation maximization (EM) (see: A. Dempster, N. Laird, and D. Rubin.Maximal likelihood form incomplete data via the EM algorithm.Proceedings of the Royal Statistical Society, 39:1-38, 1977). Therefore,p_(in)(I;ψ_(in)), p_(out)(I;ψ_(out)) may be calculated in each iterationbased on the updated intensity distribution parameters.

EXPERIMENTAL RESULTS

The suggested techniques may be exemplified on the segmentation ofcerebral hemorrhages (CH) in human brain CT scans. CH volume estimatesprovided by the segmentation may be critical to the determination of thetherapy procedure which may involve surgery in addition to medicineintake. It is important to note that CHs have well defined intensityrange, expressed in Hounsfield units (CT numbers). Nevertheless, similargray-level values as a result of calcification or proximity to the skullbone may fail a fully automatic segmentation process and therefore mayrequire the insight of a physician. The disclosed user-interactivesegmentation was tested by two radiology experts and compared with acommercial toolbox. The advantage of the suggested techniques wasobvious, considering both operating convenience (user interaction) andaccuracy (user satisfaction). Both initialization and the first stage ofthe segmentation process were fully automatic. As expected, not morethan a single or a couple of user interaction steps were needed tocomplete the segmentation. High Dice scores were obtained in acomparison with independent, fully manual annotations.

An exemplification of the disclosed techniques on cerebral hemorrhages(CH) segmentation of brain CT scans is herein disclosed. Segmentation isnecessary for an accurate estimate of the hemorrhage volume for furthermedical treatment. Scans were acquired with Philips Brilliance CT 64system without radio contrast agents injection. The data resolution is512×512×[90−100] with voxel size of 0.48 millimeter (mm)×0.48 mm×3 mm,with 1.5 mm overlap in the axial direction. 15 cases of CH seizures weretested.

Reference is now made to FIGS. 3A-3D, which show various views (i.e.,axial, coronal, sagittal and 3D) of a CT 3D medical image of CH in ahuman brain of various patients (patient 1, patient 2, patient 3 andpatient 4, correspondingly) segmented according to the disclosedprobabilistic model. FIGS. 3A-3D present qualitative results of some ofthe tested cases. The fully automated segmentation (i.e., according tostep 110 of the method of FIG. 1) is indicated in white dots (i.e., by adotted boundary). The user input (i.e., user clicks according to step120 of the method of FIG. 1) in indicated by white plus signs for falsepositive (i.e., indicating areas to be removed from the segmented ROI)and by white asterisks for false negative (i.e., indicating areas to beadded to the segmented ROI). The final segmentation, i.e., with userinteraction, is indicated by a white line (i.e., by a line boundary).For example, in the axial scan of patient 1 (i.e., in FIG. 3A), the useradded asterisks in an area which was not identified by the automaticsegmentation as a part of the ROI, i.e., a part of the CH. Accordingly,the dotted boundary does not include this area in the segmented ROI.However, the line boundary includes this area in the segmented ROI. Inanother example, the sagittal scan of patient 2 (i.e., in FIG. 3B) showstwo areas identified by the automatic segmentation as part of the ROI(indicated by a dotted boundary). According to the user input, the smallsegmented area on the right should not be part of the ROI (i.e., is notpart of the CH) and therefore indicated with plus signs. Accordingly,the line boundary does not include this area in the ROI.

Quantitative results, which were obtained with respect to manualannotation of clinical experts, are shown in Table 1 below. Thisincludes the Dice scores (see: L. Dice. Measure of the amount ofecological association between species. Ecology, 26(3):297-302, 1945),sensitivity, specificity, accuracy of the segmentation results obtainedby the proposed techniques.

TABLE 1 Dice, Sensitivity, Specificity and Accuracy. Averages of 15different slices from different patients. Phase vs. Score DiceSensitivity Specificity Accuracy Automatic 0.874 ± 0.034 0.864 ± 0.073 0.99 ± 0.0019 0.996 ± 0.0033 With user interaction 0.905 ± 0.027  0.87± 0.063 0.999 ± 0.0003 0.997 ± 0.0022

In some embodiments, the disclosed interactive segmentation may beapplied to medical images, which provide two dimensional (2D)information, with the required modifications. Such 2D interactivesegmentation may be required, for example, when the resolution in the Zaxis of a 3D medical image is very low or when the medical image is of2D, captured by using a 2D modality, such as X-ray scans.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium may be a tangible device that mayretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein may bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, may be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that may directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, may be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

1. A method comprising using at least one hardware processor for:receiving a three-dimensional medical image of an interior volume of apatient's body; automatically performing three dimensional segmentationon the three dimensional medical image to detect and define a region ofinterest in the three dimensional medical image, wherein the performingof the three dimensional segmentation comprises automaticallydetermining a boundary defining the region of interest; receiving aninput from a user with respect to the three dimensional segmentation ofthe three dimensional medical image, wherein the input comprises spatialinformation indicating one or more regions of disagreement in thethree-dimensional medical image with respect to the determined boundary;and updating the three dimensional segmentation of the three dimensionalmedical image based on the input received from the user, wherein theupdating comprises updating the determined boundary based on the spatialinformation to redefine the area of interest, and wherein the updatingis performed according to weights determining the relative contributionof each one of the three dimensional segmentation and the input from theuser to the updating of the three dimensional segmentation, therebydetermining the extent of influence of the input from the user on thedetermined boundary.
 2. The method according to claim 1, wherein thethree dimensional medical image is of an imaging modality type selectedfrom the group consisting of: Computerized Tomography (CT), MagneticResonance Imaging (MM) and ultrasound.
 3. The method according to claim1, further comprising using said at least one hardware processor forperforming three dimensional visualization of the segmented region ofinterest.
 4. The method of claim 1, further comprising using said atleast one hardware processor for estimating at least one measurablefeature of the region of interest, wherein the at least one measurablefeature is selected from the group consisting of: volume, length, width,height, pose, location and texture.
 5. (canceled)
 6. The method of claim1, wherein the receiving of the input from the user and the updating ofthe three dimensional segmentation are performed in an iterative manner.7. The method of claim 1, wherein the receiving of the input from theuser is performed via a Graphical User Interface (GUI), and wherein theGUI presents to the user one or more views of the 3D medical imageselected from the group consisting of: slice views in one or more bodyplanes and a 3D view.
 8. (canceled)
 9. The method of claim 7, whereinthe spatial information comprises anchors indicated by the user via apointing device on the one or more views of the 3D medical imagepresented by the GUI.
 10. The method of claim 1, wherein theautomatically performing of the three dimensional segmentation, and theupdating of the three dimensional segmentation, comprise using aprobabilistic model.
 11. The method of claim 10, wherein theprobabilistic model is based on Gaussian Mixture Model (GMM), andwherein the GMM is used to model a distribution of the intensities ofthe 3D medical image within and outside of the region of interest. 12.The method of claim 10, wherein the performing of the three dimensionalsegmentation further comprises defining a cost functional, the costfunctional comprising: i) an image likelihood term representing thethree dimensional segmentation, and ii) a user interaction termrepresenting the input from the user, assigning the weight of theweights determining the relative contribution of the three dimensionalsegmentation to the image likelihood term, assigning the weight of theweights determining the relative contribution of the input from the userto the user interaction term, and setting the weight of the weightsassigned to the user interaction term to zero.
 13. The method of claim12, wherein the performing of the three dimensional segmentation and theupdating the three dimensional segmentation further comprise finding asegmentation that optimizes the cost functional by using a gradientdescent process.
 14. (canceled)
 15. A medical system comprising: amedical imaging device; at least one hardware processor configured to:i) automatically perform three dimensional segmentation on the threedimensional medical image to detect and define a region of interest inthe three dimensional medical image, wherein the performing of the threedimensional segmentation comprises automatically determining a boundarydefining the region of interest; ii) receive an input from a user withrespect to the three dimensional segmentation of the three dimensionalmedical image, wherein the input comprises spatial informationindicating one or more regions of disagreement in the three-dimensionalmedical image with respect to the determined boundary; and iii) updatethe three dimensional segmentation of the three dimensional medicalimage based on the input received from the user, wherein the updatingcomprises updating the determined boundary based on the spatialinformation to redefine the area of interest, and wherein the updatingis performed according to weights determining the relative contributionof each one of the three dimensional segmentation and the input from theuser to the updating of the three dimensional segmentation, therebydetermining the extent of influence of the input from the user on thedetermined boundary.
 16. The medical system according to claim 15,wherein said medical imaging device is selected from the groupconsisting of: a Computerized Tomography (CT) device, a MagneticResonance Imaging (MRI) device and an ultrasound imaging device.
 17. Themedical system according to claim 15, wherein said at least one hardwareprocessor is further configured to perform three dimensionalvisualization of the segmented region of interest.
 18. The medicalsystem of claim 15, wherein said at least one hardware processor isfurther configured to estimate at least one measurable feature of theregion of interest, wherein the at least one measurable feature isselected from the group consisting of: volume, length, width, height,pose, location and texture.
 19. (canceled)
 20. The medical system ofclaim 15, wherein the receiving of the input from the user and theupdating of the three dimensional segmentation are performed in aniterative manner.
 21. The medical system of claim 15, wherein thereceiving of the input from the user is performed via a GUI, and whereinthe GUI presents to the user one or more views of the 3D medical imageselected from the group consisting of: slice views in one or more bodyplanes and a 3D view.
 22. (canceled)
 23. The medical system of claim 22,wherein the spatial information comprises anchors indicated by the uservia a pointing device on the one or more views of the 3D medical imagepresented by the GUI.
 24. The medical system of claim 15, wherein theautomatically performing of the three dimensional segmentation, and theupdating of the three dimensional segmentation, comprise using aprobabilistic model, wherein the probabilistic model is based onGaussian Mixture Model (GMM), and wherein the GMM is used to model adistribution of the intensities of the 3D medical image within andoutside of the region of interest.
 25. (canceled)
 26. The medical systemof claim 24, wherein the performing of the three dimensionalsegmentation further comprises defining a cost functional, the costfunctional comprising: i) an image likelihood term representing thethree dimensional segmentation, and ii) a user interaction termrepresenting the input from the user, assigning the weight of theweights determining the relative contribution of the three dimensionalsegmentation to the image likelihood term, assigning the weight of theweights determining the relative contribution of the input from the userto the user interaction term, and setting the weight of the weightsassigned to the user interaction term to zero. 27.-40. (canceled)